Analysis of Su1Vx7

Analysis / figures of results from h_DsRnn_Prep_spa.ipynb
Summarizing mult. RNNs / data sets based on Pandas ExTb and ProcTb.

PREC: h_DsRnnAna_Pd.iphnb
jupyter nbconvert --TemplateExporter.exclude_raw=True --TemplateExporter.exclude_input=True --to html --output tmp h_DsRnnAnaPd_Su1Vx7.ipynb

jupyter nbconvert h_DsRnnAnaPd_Su1Vx7.ipynb \
--to html \
--output tmp \
--TemplateExporter.exclude_raw=True \
--TemplateExporter.exclude_input=True \
--TagRemovePreprocessor.remove_cell_tags="{'rmCell'}" \
--TagRemovePreprocessor.remove_all_outputs_tags="{'rmOut'}" \
--RegexRemovePreprocessor.patterns="[r'(?ms).*DeprecationWarning.*']" \
--RegexRemovePreprocessor.patterns="[r'(?ms)Using TensorFlow.*']" \

Add this cell metadata (JSON) e.g: { "tags": ["rmOut"]}

Imports

MatplotlibDeprecationWarning: 
The examples.directory rcparam was deprecated in Matplotlib 3.0 and will be removed in 3.2. In the future, examples will be found relative to the 'datapath' directory.
   /usr/local/Anaconda/envs/py3.6/lib/python3.6/site-packages/ipykernel_launcher.py:4
  after removing the cwd from sys.path.

ProcTb: Load model parameter sets

Out[3]:
(36, 112)
Out[4]:
ProcId                                      Su1Vx7_Su003Vx2
DataDir        /data/mandelkowhc/Sleep/0401_Proc05_8su_Ap3/
SuId                                                    003
NiiFile                                         errts*.BRIK
NiiFileNr                                              None
NiiMaskFile                                   Epi_mask.nii*
Fs                                                       20
TR                                                        3
TRFS                                                     60
CropData                                               3600
NX                                                        3
NY                                                        1
NT                                                      300
NB                                                        1
Tbatch                                                  240
MaskVal                                                   0
Drop                                            (0.5, last)
ValDrop                                         (0.5, last)
Xtrafo                                               [step]
Xrnd                                                   None
Xnul                                                   None
RandXlead                                                 0
ValSu                                                  None
ValEx1                                                    2
ValRank                                               False
ValMiddle                                              True
ValTag                                                    2
Xmc                                                   False
GsigFile                                           Gsig.mat
RnnNio                           [4, G, 128, 128, 64, D, 1]
Epochs                                                  150
ValFrac                                                None
ProcIds                                                  []
SbJob                                                  None
SbTask                                                    0
ProcInfo                                                   
Name: Su1Vx7_Su003Vx2, dtype: object

[ Load ExTb - need this? ]

Load models / ProcDirs

Out[6]:
149
Proc_Su1Vx7/Km_Su1Vx7_Su003Vx1/
Proc_Su1Vx7/Km_Su1Vx7_Su003Vx2/
Proc_Su1Vx7/Km_Su1Vx7_Su003Vx3/
Proc_Su1Vx7/Km_Su1Vx7_Su003Vx4/
Proc_Su1Vx7/Km_Su1Vx7_Su003Vx5/
Proc_Su1Vx7/Km_Su1Vx7_Su003Vx6/
Proc_Su1Vx7/Km_Su1Vx7_Su003Vx7/
Proc_Su1Vx7/Km_Su1Vx7_Su006Vx1/
... total 56
........................................................CPU times: user 977 ms, sys: 1.66 s, total: 2.63 s
Wall time: 1min 32s

Convert Pars to Pandas (ProcTb or SecTb)

Out[9]:
dict_keys(['ProcDir', 'Xtrafo', 'CropData', 'DataChan', 'DataDir', 'Drop', 'Epochs', 'Fs', 'GsigFile', 'MaskVal', 'NB', 'NT', 'NX', 'NY', 'Ngpus', 'NiiFile', 'NiiFileNr', 'NiiMaskFile', 'ProcId', 'ProcIds', 'ProcInfo', 'RnnArg', 'RnnNio', 'SbJob', 'SbTask', 'SuId', 'TR', 'TRFS', 'Tbatch', 'ValDrop', 'ValEx1', 'ValFrac', 'ValMiddle', 'ValSu', 'ValTag', 'Xmc', 'Xnul', 'Xrnd', 'TrainLog', 'SecNp', 'SecTb', 'x', 'y', 'yh'])

Check training success

RuntimeWarning: Degrees of freedom <= 0 for slice
   /usr/local/Anaconda/envs/py3.6/lib/python3.6/site-packages/numpy/core/_methods.py:140
  keepdims=keepdims)
RuntimeWarning: invalid value encountered in true_divide
   /usr/local/Anaconda/envs/py3.6/lib/python3.6/site-packages/numpy/core/_methods.py:110
  arrmean, rcount, out=arrmean, casting='unsafe', subok=False)
RuntimeWarning: invalid value encountered in double_scalars
   /usr/local/Anaconda/envs/py3.6/lib/python3.6/site-packages/numpy/core/_methods.py:132
  ret = ret.dtype.type(ret / rcount)
RuntimeWarning: divide by zero encountered in double_scalars
   /usr/local/Anaconda/envs/py3.6/lib/python3.6/site-packages/ipykernel_launcher.py:4
  after removing the cwd from sys.path.

Compare training success across model parameters (ProcGroups)

Notes:

  • Su1Vr3 and Su1Vr3lin show little difference in average TrainMetric
    • ...except for 3 neg. outliers (better) in the former.
Out[13]:
TrainMetric BestValEpoch
ProcGroup ProcId
Su1Vx7 Su006Vx6 -11.396064 59
Su056Vx6 -inf 2
Su078Vx3 -10.107036 101
Su078Vx6 -10.808949 136
Su105Vx1 -inf 2

Make Xarray from Pars (~SecTb)

Out[14]:
(711, 4800, 6)
Out[19]:
Coordinates:
  * Ch       (Ch) <U4 'card' 'resp' 'mask' 'Gsig' 'y' 'yh'
  * Sec      (Sec) MultiIndex
  - Su       (Sec) int64 3 3 3 3 3 3 3 3 3 ... 105 105 105 105 105 105 105 105
  - Se       (Sec) int64 2 2 2 2 2 2 1 1 1 1 1 1 1 ... 1 1 1 1 1 1 1 1 1 1 1 1 1
  - Ex       (Sec) int64 12 12 12 12 12 12 7 7 7 7 7 7 ... 8 8 8 8 8 8 8 8 8 8 8
  - Sc       (Sec) int64 0 1 2 3 4 5 0 1 2 3 4 ... 7 8 9 10 11 12 13 14 15 16 17
  * t        (t) float64 0.0 0.05 0.1 0.15 0.2 ... 239.8 239.8 239.8 239.9 239.9
    Clipped  (Sec) bool True False False True False ... False False False False


Compute Metrics

Out[29]:
array([0.692, 1.06 , 0.802, 0.777, 1.053, 0.712, 0.943, 0.937, 0.681,
       0.809, 0.75 , 0.67 , 0.885, 0.985, 0.882, 1.027], dtype=float32)

Boxplot Metric stats

Table of median per section

Out[33]:
Clipped WRVF WRVFa WRVFz Pcorr Rcoef
Su Se Ex Sc
3 1 3 0 1.0 0.842806 0.647232 0.981324 0.143 0.406
1 1.0 0.825933 0.868798 0.864757 0.368 1.248
2 1.0 0.820356 0.643844 1.004619 0.026 0.093
3 1.0 0.782056 0.648811 0.995290 0.070 0.505
4 1.0 0.672198 0.437216 1.023237 -0.043 -0.158
... ... ... ... ... ... ... ... ... ...
105 2 14 11 1.0 0.993318 1.234349 0.831725 0.415 3.332
12 0.0 0.835396 1.064900 0.800054 0.504 4.041
13 1.0 0.894149 0.653225 1.062293 0.178 1.016
14 1.0 1.004728 0.877484 1.097575 0.201 1.782
15 1.0 0.680804 0.604457 0.880927 0.401 1.129

711 rows × 6 columns

Which Su data best predicts it's own left-out scans?

Out[35]:
WRVFz Pcorr Rcoef
Su
65 0.764530 -0.504 -2.9340
63 0.875602 -0.377 -2.2480
78 0.920989 -0.310 -1.5340
6 0.940632 -0.322 -1.8630
3 0.948437 -0.300 -1.5370
105 0.952010 -0.273 -1.5335
86 0.964542 -0.213 -1.1350
56 0.995946 -0.149 -0.8805

Which experiments are best predicted by each subject's data?

Out[36]:
WRVFz Pcorr Rcoef
Su
3 (3, 2, 12) (3, 2, 12) (3, 2, 12)
6 (6, 1, 5) (6, 1, 5) (6, 1, 5)
56 (56, 1, 7) (56, 1, 7) (56, 1, 7)
63 (63, 2, 12) (63, 2, 12) (63, 2, 12)
65 (65, 1, 5) (65, 2, 12) (65, 1, 5)
78 (78, 2, 9) (78, 2, 9) (78, 2, 9)
86 (86, 1, 9) (86, 1, 9) (86, 1, 7)
105 (105, 2, 12) (105, 2, 12) (105, 2, 10)

Plot best signals

Top sections for 3 metrics


Top 12 sections for Pcorr metric

Top 3 Pcorr for each Subject




Carpet plots?

Out[40]:
(711, 80)
Out[112]:
array([  3,   6,  56,  63,  65,  78,  86, 105])


Scratch

Looks like Su 65 and 6 (or 78?) show the best results on these metrics.